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ANY TREE 



MATEUSZ MICHALEK 

Abstract. In this paper we present geometric features of group 
based phylogenctic models. We address a long standing problem of 
determining the ideal of the claw tree jSSOSj , |DK09| . We focus on 
the 3-Kimura model. In particular we present a precise geomet- 
ric description of the variety associated to any tree on a Zariski 
open set. This set contains all biologically meaningful points. The 
result confirms the conjecture of Sturmfels and SuUivant jSSOSj 
on the degree in which the ideal associated to 3-Kimura model is 
generated on that set. 



1. Introduction 

To a tree T and an abelian group G one can associate an algebraic 
variety X{T,G). The construction of this variety is motivated by bi- 
ology. The tree T describes the evolution of species and the group G 
distinguishes a model of evolution. By a model of evolution we mean 
certain constraints on the probabilities of mutation. In the article we do 
not consider the details of biological aspects of this construction. They 
are well described in |PS05] . or in a short paper |ERSS04] . Our main 
object of study is the variety X(T, G) and its geometry. Nevertheless 
some of the applications of the results will be suggested. Moreover the 
case of the group ^ = ^2X^2 will be of particular interest. This group 
corresponds to the 3-Kimura model appearing in theoretical biology. 

From the point of view of algebraic geometry the variety X(T, G) 
has got a lot of interesting properties. The first results are presented 
in |ES93] and |SSE93] . In the modern mathematical language, the 
authors prove that X{T, G) is toric. We want to stress that throughout 
the article we do not assume that a toric variety is normal. Directly 
speaking X(T, G) is the closure of the image of an algebraic map given 
by monomials. 

One of the main advantages of toric geometry is its interaction with 
discrete geometry. The use of lattices, polytopes and fans allows to 
precisely describe toric varieties and do exact computations |Ful93j . 
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[CLS] . The varieties X[T,G) also have a lot of connections with dis- 
crete objects - trees, groups, polytopes. Hence, the study of them often 
involves combinatorics, toric geometry or even representation theory. 
One of the examples is presented in |BW07] where the binary model 
G = Z2 is studied. The major result presented there says that if two 
trees Ti and T2 have the same number of leaves, then the varieties 
X(Ti, Z2) and X(T2, Z2) belong to the same flat family. Unfortunately 
this is a special feature of the group Z2 [KublOj . [DBMt Section 3.2]. 
One can also find deep connections with the representation theory. 
The algebras of the considered varieties are connected with algebras of 
conformal blocks |Man09j . Further interesting results can be found in 
[SXTO] . [BuclOj . jXROS] . jDK09] and [SuIOT] . 

The 3-Kimura model is much more complicated then the binary 
model. In particular the Hilbert polynomial of the associated vari- 
ety depends on the topology of the tree, not only the number of leaves. 
Still this model is one of the simplest that are of interest to biologists. 

The main aim of this article is to present geometric constructions 
that lead to a better understanding of the variety X{T, Z2 x Z2). One 
of the motivations is the conjecture of Sturmfels and Sullivant. It claims 
that the ideal of X{T, Z2 x Z2) is always generated in degree 4 [SS05[ 
Conjecture 2]. This conjecture seems more algebraic than geometric. 
However it can be presented in a purely geometric language in terms 
of scheme-theoretic intersection of varieties. Let Ki^i be the claw tree 
with one inner vertex and / leaves. 

Conjecture 1.1 (Conjecture 4.6 [DBMj . cf. Conjecture 2 [5505] ). The 

variety X{Ki^i,7j2 x Z2) is a scheme-theoretic intersection of the va- 
rieties X(T, Z2 X Z2) , where T is any tree with I leaves different from 

The above conjecture can be stated for any phylogenetic model. It 
could provide the description of the ideal of claw trees for many relevant 
models. The reduction to claw trees for the general Markov model 
was presented in |AR08] and for equivariant models in |DK09j . The 
application of the results for 3-Kimura model is presented in Section O 
To stress the importance of the conjecture we cite [DK09j : 

"We have now reduced the ideals of our equivariant models to those 
for stars, and argued their relevance for statistical applications. The 
main missing ingredients for successful applications are equations for 
star models. These are very hard to come by (...)"• 

Let us fix a coordinate system on A" D X(T, Z2XZ2). Let B C A" be 
the Zariski open subset containing points with all coordinates different 
from zero. In this paper we prove that Conjecture 11.11 holds on B. 
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It follows that Conjecture 2 of jSSOSj holds after a localization of the 
algebra with respect to all the coordinates. In particular all of the 
previous conjectures hold if and only if the intersection of the varieties 
X(T, Z2 X Z2) is reduced and irreducible. The main idea of the proof 
is to transfer the properties of X{T, Z2) to X{T, Z2 x Z2). This is done 
by an interplay between toric geometry and combinatorics. The best 
reference for such techniques is |Stu96j . 

Among the applications we give all biologically meaningful invariants 
for the 3-Kimura model on any tree. We also discuss the identifiability 
issue. The main aim of the paper is to interest toric geometers and al- 
gebraic combinatorialists in the arising problems. Hence, we postpone 
the discussion of applications to Section [61 

Let us briefly discuss the structure of the paper. Section [2] settles the 
notation for the rest of the paper. All introduced notions are formal 
mathematical objects and much of the biological motivation is skipped. 
The most important are Definitions 12.61 and 12.81 that are far from being 
standard in algebraic phylogenetics. Therefore an expert in the disci- 
pline is advised at least to take a look at these two definitions. People 
with mathematical background and no experience in biology should be 
able to follow this sectiorQ and are encouraged to do so. 

In Section [3] we introduce objects typical of toric geometry. Some of 
the presented facts may be well known to toric geometers. We include 
them as an introduction to the proof of the main theorem. We also 
explain special features of varieties X[T,G) and interactions between 
them. 

Section|l]describes relations between groups G and varieties X{T, G). 
We try to avoid general theory that will not be needed. Instead, we 
introduce basic concepts that are later used directly to prove the main 
theorem. We are interested in the interactions between morphisms 
of groups and morphisms of varieties. We are not setting a general 
categorical environment for phylogenetic models. This can be done, 
but involves more sophisticated language and is not required for the 
rest of the paper. The restriction to the case where G is abelian allows 
to obtain short, natural statements. 

In Section O we bring all the ideas together to prove the main the- 
orem. The following Section [HI discusses some of the possible applica- 
tions. In fact, the dense open subset B contains all points interesting 
form the biological point of view |CFS08] . This section should be par- 
ticulary important to people interested in algebraic phylogenetics. It 
is self contained and results presented there are not used in the rest of 
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the paper. However in this section we allow ourselves to use standard 
notions from phylogenetics without introducing them. 

The last Section [7] describes some partial results concerning the com- 
plement of -B. It is quite technical. It contains a few results in favor of 
Conjecture 11.11 In Appendix we present two results about the binary 
model. 

Acknowledgements 
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me to the subject and guiding me through it. I would also like to thank 
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2. Notation 

We will be dealing with algebraic varieties associated to phylogenetic 
models. These varieties are always given as closures of the image of a 
parametrization map. A short introduction to the topic can be found 
in |ERSS04] . In this paper we will be interested in the geometry of al- 
gebraic varieties. Hence we omit their detailed construction motivated 
by biology. It can be found for example in |MiclOt Section 2], |SS05] 
or in a nice theoretical setting in |DK09] . We briefly present a purely 
algebraic construction of varieties associated to G-models. The proofs 
of these results can be found in |MiclOj . We present them to make 
the paper self-contained for any algebraic geometer with a limited or 
no knowledge of biology. We hope that arising geometric problems will 
interest toric geometers, even if they are not dealing with phylogenetics. 

We will be defining objects that will depend on a tree T and a group 
G. For any object O if we want to stress its dependence on either T or 
G we write them in the upper index: O"^'*^. 

2.1. A variety associated to a G-model. Let S* be a finite set. 

Definition 2.1 (Space W). We define W to be a complex vector space 
spanned freely by elements of S. More precisely W = (Baes'^a, where Cq 
is a field of complex numbers corresponding to one dimensional vector 
space spanned by a E S. 

Suppose that a group G acts on the set S, hence also on the vector 
space W. This induces a G action on End(H^). For / G End(iy), 
g E G and x E W we have {gf){x) = g{f{g~^x)). 

Definition 2.2 (Space W). We define W C End(Vr) as the maximal 
space of endomorphisms invariant with respect to the G action. 
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Let us consider the following setting. Assume that G contains a 
normal, abelian subgroup H that acts transitively and freely on the set 
S. This is a technical assumption. It is crucial however to prove that 
the varieties we obtain are toric. Models obtained in this way are called 
G-models |MiclO] . |BDW09] . The majority of the paper concerns the 
binary model and the 3-Kimura model. They both correspond to the 
case when G is abelian. 

Definition 2.3 (General group based model). We will say that we are 
dealing with a general group-based model if G is abelian. Moreover we 
assume that S is a torsor for G. In this setting the vector space W is 
the regular representation of G. 

Remark 2.4. Models coming from biology do not have to be given 
by a group action. However the class of G-models is the largest class 
known to the author that gives rise to toric varieties. 

In the following discussion we assume that we are dealing with a 
general group-based model. Using more complicated language similar 
results can be obtained for G-models |MiclO] . The restriction to abelian 
groups in particular allows us to work with group elements, instead of 
sets of characters. Suppose that we are given a rooted tree T with 
edges directed from the root. 

Definition 2.5 (Sets L, V , N and E). Let L, V, N and E be respec- 
tively the set of leaves, vertices, nodes and edges of the tree T. We have 
V = L U N . We identify leaves with edges adjacent to them. 

Definition 2.6 (Socket). A socket is a function s : L ^ G such that 
YlieL^i^) ~ ^> "^here is the neutral element of the group G. 

Example 2.7. Let us consider the group G = Z3 and the following 
tree: 



Here 62, 63, 64 and 65 are leaves. An example of a socket is an associa- 
tion €2 — )■ 1, 63 — )■ 1, 64 — )■ 2, 65 — 7- 2. 

Suppose that we have associated group elements to edges adjacent 
to a node n & N . We say that a signed sum around this node is 
trivial if the element associated to the incoming edge equals the sum of 
elements associated to outcoming edges. For the root we want the sum 
of elements associated to outcoming edges to be the neutral element. 
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Definition 2.8 (Network). A network is a function f : E ^ G such 
that the signed sum of associated elements around each node is trivial. 

Example 2.9. We consider the same tree as in Example 12. 7[ We can 
make a network using the same association and extending it by ei — )> 2. 

Proposition 2.10. There is a natural bijection between sockets and 
networks. 

Proof. Let n be a network. We know that the signed sum S{v) around 
each inner vertex v is the neutral element element. Hence YlveN ^i'^) ~ 
e, where e is the neutral element. Let us consider an edge directed from 
vi to V2, where vi,V2 € A^. Let us note that the group elements n{vi, V2) 
and n{vi,V2)~^ appear in S{vi) and S{v2)- We see that J2veN ^i"") ~ 
Yli^L '^(O- This means that a restriction of the network to leaves gives 
a socket. 

Given a socket s we can define a function n : E ^ G using the 
condition of summing up to the neutral element around inner edges. 
The only nontrivial thing is to notice that the sum around the root 
also gives the neutral element. This follows from the previous equality 
Yliv^N ^i'^) ~ XlzeL ^(0 the fact that S{v) = e for each node v 
different from the root. □ 

To each edge e G -E we associate a vector space = W . It is an easy 
exercise that We is IGl-dimensional. We assume that the coordinates 
of Wf. are indexed by group elements. In the same way to each leaf I we 
associate a vector space Wi = W with coordinates indexed by elements 
of G. 

Definition 2.11 (Spaces We and Wl)- We define the following spaces: 
Wl = (^Wi, WE = (^We. 

We call Wl the space of states of leaves and We the parameter space. 

An element of a base of Wl can be given by a tensor product of base 
elements of each Wi. The base of Wi corresponds to group elements. 
Hence elements of the base of Wl correspond to associations of group 
elements to leaves. In the same way elements of the base of We are 
represented by associations of group elements to edges. We define two 
subspaces spanned by subsets of distinguished basis. 

Definition 2.12 (Spaces We, Wl)- The subspace We C We is spanned 
by basis elements corresponding to associations that form a network. 
The subspace Wl C Wl is spanned by basis elements corresponding to 
associations that form a socket. 
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Thanks to Proposition [2]T0] we have got a natural isomorphism We = 
Wl- We have got the following map: 

The first arrow is the tensor product and the second is the projection 
onto the subspace. The induced projective maps are the following: 

: Yl ¥{We) ^ nwE) nwL). 

The first map is the Segre embedding and the second is a projection. 
Note that it follows from definition that the maps ip and ipp are given by 
monomials in the given basis. The closure of the image of ipp (resp. ip) 
is the projective (resp. affine) variety associated to the model. This 
is the main object of our study. We will denote it by F{X{T,G)) 
(resp. X{T,G)). 

3. Basic properties of algebraic varieties associated to 

phylogenetic models 

3.1. Toric varieties. First we have to point out that we do not assume 
that a toric variety has to be normal. We only assume that a torus 
acts on a variety and one of the orbits is dense. This setting is most 
common when dealing with applications. Much information can be 
found in |Stu96] . The main drawback of this approach is that the 
varieties we consider will not be given by a fan. However, still they can 
be represented by polytopes, that do not have to be normal. For this 
reason we will often work with the character lattice M instead of the 
one parameter subgroup lattice A^. 

Each vector space can be considered as a toric variety. We fix a 
coordinate system. The set of points with nonzero coordinates is an 
algebraic torus. It acts on the vector space. Moreover the dense torus 
orbit can be identified with the torus. As each vector space we have 
considered so far had a distinguished basis we can associate to it the 
character lattice of a torus. 

Definition 3.1 (Lattices Ms, Me, Me)- For en edge e & E the lattice 
Me is the character lattice of the torus acting on We- The lattice Me 
is the character lattice of the torus acting on YleeE -^^^ lattice Ms 
is the character lattice of the torus acting on We = W^. 

Let us note that the coordinate system on the vector space distin- 
guishes the basis of the lattice. The basis of each lattice Me is indexed 
by group elements. As Me = ^eeE the basis of Me is indexed 
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^(e,3)(/« 



by pairs {e,g) where e is an edge and g a group element. The basis 
elements of Ms corresponds to sockets or networks. The morphism 
ip '■ riee-B — ^ We is a toric morphism. 

Definition 3.2 (Morphism ijj). The morphism ijj : Ms — > Me is the 

morphism of lattices induced by ip. 

In this setting the description of i/j is particulary simple. Let fn G Ms 
be a basis vector corresponding to a network n. The element ip{fn) will 
be an element of the unit cube in Me- Let /i(e,g) G Me be the basis 
vector indexed by a pair {e, g) E E x G and let /i^^ be its dual. We 
have: 

1 if n(e) = g 
otherwise. 

We come to the most important definition of this section. 

Definition 3.3 (Polytope P). We define the polytope P C Me to he 
the image of the basis of Ms hy 'ip. In other words the vertices of the 
polytope P correspond to networks. More precisely each vertex has got 
1 on coordinates indexed by pairs that form a network and on other 
coordinates. 

Example 3.4. Let us consider the tree T with one inner vertex and 
three leaves li, I2 and Z3. Let G = Z2. The lattice Ms is the 4 dimen- 
sional lattice generated freely by vectors e(o,o,o)) ^(i^i^o)) 6(1^0,1)) ^(0,1,1) 
that correspond to sockets/networks on T. The lattice Me is a 6 di- 
mensional lattice with basis vectors f{ii,g) with 1 < z < 3 and g E 1^2- 

We have iJ^e^aAc)) = f(h,a) + f(i2,b) + f(h,c)- Hence each vertex of P 
will have three coordinates equal to zero and three to one. Let us con- 
sider the base of Me in the following order /(^^^o), • • • , f{hfl)J(h,i)- 
The vertex corresponding to e(o,o,o) is (1,0,1,0,1,0). In the same or- 
der 6(1,1,0) (0,1,0,1,1,0), 6(1,0,1) (0,1,1,0,0,1) and 6(0,1,1) 
(1, 0, 0, 1, 0, 1). These are of course all vertices of P. 

The polytope P is the polytope associated to the toric variety X(T, G). 
The algebra of this variety is the algebra associated the semigroup gen- 
erated by P in Me- Note that P does not have to generate the lattice 
Me- 

Definition 3.5 (Lattice Me)- We define the lattice Me as a sublattice 
of Me generated by P. 

The lattices defined so far corresponded to affine objects. A rational 
map from a vector space to its projectivization is well defined on points 
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with non zero coordinates. Hence it induces a surjective morphism of 
tori, what corresponds to an injective morphism of character lattices. 

Definition 3.6 (Degree functions degg). Note that for a character lat- 
tice M with a distinguished basis we can define a function deg : M — t- Z 
that sums up coordinates. The degree of a lattice element is the degree 
of the monomial function associated to it. For lattices Me the corre- 
sponding degree functions are denoted by degg. 

Definition 3.7 (Lattices M^q, and Mea). For a lattice Ms we 

define Ms a as a sublattice of elements with the sum of coordinates equal 
to zero. In particular Msfi is the character lattice of the torus acting 

on F{We). We define Me a as a sublattice of Me defined by equalities 
degg = for each e. This is the character lattice of the torus acting on 
]\¥{We). We also define Mea := Mea n Me- This is the character 
lattice of the torus acting on the projective toric variety F{X{T)). 

3.2. Intersection of tori. To prove the main theorem we will also 
need to briefly review the theory of intersection of tori. Suppose that 
we have got tori Tj that are subtori of a torus T. Let Mj be the character 
lattice of the torus Tj and M the character lattice of the torus T. The 
inclusion Ti ^ T gives a surjective morphism M — )■ Mj. The kernel 
Ki of this morphism corresponds to characters of T that are trivial on 
Tj. In the algebra of T the ideal of Tj is generated by elements of the 
type k — 1 for k G Ki. Let T' be the scheme theoretic intersection of 
the tori Ti. Let M' be a sublattice of M generated by the lattices K^. 

Fact 3.8. The algebraic set T' is irreducible iff M' is saturated. The 
components of T' are tori and the number of components equals the 
index of M' in its saturation in M. If M' is saturated then M/M' is 
the character lattice of T' . The functions k — 1 for k G M' generate 
the ideal of T' . 

3.3. Interactions between trees and varieties. We can define an 
order on trees with / leaves as follows. We say that Ti < T2 if Ti can be 
obtained form T2 by a series of contractions of inner edges. Here by an 
edge contraction we mean identifying two vertices of a given edge. The 
smallest tree with / leaves is the claw tree Ki^i with one inner vertex. 
This is a part of a construction of the tree space [BHVOlj . We fix an 
abelian group G. 

Proposition 3.9. //Ti < T2 then X{Ti,G) C X(T2,G). 

Proof. Although the statement is very easy we believe that the fol- 
lowing discussion may be helpful to better understand the rest of the 
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paper. Both trees have got the same number of leaves, so we can make 
a natural bijection between their sockets. This gives an isomorphism 
of the ambient spaces We- As Ti < T2 we can make an injection 
from the edges of Ti to the edges of T2. Note that a network on T2, re- 
stricted to the edges of Ti is a network on Ti. This gives us a projection 
vr : M^^ M'^^. The map vr simply forgets the coordinates indexed 
by {e,g), where e is an edge of T2 not corresponding to an edge of Ti. 
Moreover the projection of P^^ is equal to P^^. The following diagram 
commutes: 



Ml' 



Ms 




Any relation between the vertices of P^'^ is also a relation between the 
vertices of P^^ . Hence any polynomial in the ideal of X{T2,G) is also 
in the ideal of X(Ti,G). □ 

The surjective morphism of algebras corresponding to the inclusion of 
varieties is given by the restriction of the surjective morphism between 
and M^' to the cones spanned by polytopes P^"^ and P^^. 

It is natural to ask what is the relation between X{To,G) and the 
scheme theoretic intersection of all X{T, G) for Tq < T. The first 
conjecture is that if there exists at least one T > Tq, then they are 
equal. An equivalent conjecture was stated in jPBMj . 

Conjecture 3.10. The ideal of the variety associated to the claw tree 
Ki^i is the sum of all the other ideals associated to trees with I leaves. 

So far we only know that the answer is positive for G = Z2 |CP07] . 
|SS05] . |DBM] . In the case particulary interesting for biologists G = 
Z2 X Z2 we know that this conjecture implies Conjecture 2 form [SS05] 
and is equivalent to it for / > 8. For other groups Conjecture 13.101 is 
equivalent to Conjecture 1 from |SS05j for / large enough. 

Conjecture 13.101 can be stated for any phylogenetic model, not nec- 
essarily given by a group. In particular for a general Markov model. 
One would be also interested to know exactly what is an intersection 
of a few varieties associated to different trees. In particular how many 
ideals do we have to sum to obtain the ideal associated to the claw 
tree. For the binary model some answers can be found in [DBM] . A 
stronger result can be found in Appendix 18. 1[ One could also hope that 
the intersection of X(Ti,G) and X{T2,G) is equal to X(T,G) where 
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T is the largest tree smaller then Ti and T2. A counterexample to this 
statement is presented in Appendix 18.21 

4. Category of general group based models 

In this section we will establish connections between morphisms of 
groups and morphisms of corresponding varieties. Once again our main 
aim is application in geometry. We are building the set up of the next 
section. That is why we restrict to special cases. This reduces the 
complexity of the language but still gives a geometric insight. Let us 
fix a tree T. 

Definition 4.1 (Groups & and By edgewise action sockets and 
networks on a given tree form a group. These groups will he denoted 
respectively by & and DT. 

Let / : Gi — 6*2 be a morphism of abelian groups. It induces 
morphisms (S*^^ — ?■ and Dl*^^ — ?> DT*^^. This gives the following 
commutative diagram : 

(4.1) 



MP — ^M^^ 

Hence the morphism M^^ — )■ of character lattices restricts to 

cones over polytopes. This gives a morphism of algebras of associated 
varieties. The morphism M^^ — )■ restricts to positive quadrants of 
both lattices. Hence we get a morphism of ambient spaces / : W^'^ — ?■ 

W^^ compatible with morphism of varieties / : X(T, G2) X{T,Gi). 
This gives a contravariant functor from the category of abelian groups 
to the category of embedded affine toric varieties. Moreover if / is 
injective (resp. sujective) then / is dominant (resp. injective). The 
second assertion is an easy exercise. 

We also need the following setting. Suppose that we have morphisms 
/i : G — 7- Gj for i = 1, . . . , m. Just as above this gives us a morphism of 
embedded varieties Yl ^{T, Gi) X(T, G). If the product f\^- ■ ■y< fm 
is sujective then the morphism of varieties is injective. However in 
general if the product /i x • • • x /m is injective the morphism of varieties 
does not have to be dominant. 



12 



MATEUSZ MICHALEK 



5. Main Theorem 
Our aim is to prove the following conjecture for G = Z2 x Z2. 

Conjecture 5.1. The dense torus orbit of the toric variety X{Ki^i, G) 
is an intersection of the dense torus orbits of varieties X{T,G), where 
T is any tree with I leaves different form the claw tree. 

Note that all dense torus orbits are contained in the dense torus orbit 
O of the projective (or affine) ambient space. In the algebraic set O all 
the considered orbits are closed subschemes. Hence Conjecture 15.11 can 
be regarded in a set-theoretic or in a scheme-theoretic version. Both 
of them are equivalent. This follows for example from a more general 
statement |ES96l Corollary 2.2] and is particulary simple in toric case. 
However because the proofs of both versions are basically the same for 
G = Z2 X Z2 we have decided to include both. Moreover this also 
gives an idea how the elements of the ideal of X{Ki i, Z2 x Z2) can be 
generated by elements of ideals of X{T, Z2 x Z2). 

The main idea of the proof is to extend the results known for binary 
models to the 3-Kimura model. The binary model is very well under- 
stood and has a lot of special properties |BW07] . In particular from 
[DBM|, Theorem 4.9] we know that Conj ecture 13.101 holds for G = Z2. 

We have got three natural projections : Z2 x Z2 — )■ Z2 for i = 
1, . . . , 3. The map /i x /2 x : Z2 x Z2 — > Z2 x Z2 x Z2 is injective. 
Moreover it induces a dominant map from the product of three binary 
models onto the 3-Kimura model. This map is the key tool that will 
allow us to transfer some of the properties from the binary model to 
the 3-Kimura model. Unfortunately the map is not surjective, but just 
dominant. We can projectivise the varieties, but then we get a rational 
map. It turns out that a combine use of both of the maps allows to 
derive the main theorem. 

Let /* : Mg^""^^ M^^ be a morphism of lattices induced by fi. 
Let g : Mg^ — )■ M^^ be the morphism of lattices that corresponds to 
the parametrization map of the binary model. 

We have got the following commutative diagram similar to Diagram 




The following Fact is well known. 
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Fact 5.2. The dimension of the affine 3-Kimura model is equal to 
Sl-El + 1. The dimension of the product of three affine binary models is 
equal to 3(|i?| + 1). The dimension of the projective 3-Kimura model is 
equal to Bl-El. The dimension of the product of three projective binary 
models is equal to 3|i?|. □ 

It follows that if we consider projective varieties representing the 
models, the dominant morphism described above becomes a rational 
finite map. As the map between projective varieties is not a morphism 
we will restrict our attention only to dense orbits of the tori. On these 
tori orbits all maps are well defined and are represented by morphism 
of lattices. 

5.1. Maps of dense torus orbits. Let us consider the following di- 
agram: 

(5.1) 

X X M^^ ''—^ A-f|= X X 




First let us explain the morphism j. It is injective, as it is a re- 
striction of i. The lattice Me^ is the character lattice of the torus 
acting on the projective toric variety representing the model. The 
morphism j is induced by the rational finite map form the product of 
three P(X(T,Z2)) to P(X(T,Z2 x Z2)). Due to the coordinate system 
we can identify dense torus orbits with the tori. 

Definition 5.3 (The torus Tx)- Let X he any toric variety in an affine 
or projective space with a distinguished coordinate system. We may 
identify the dense torus orbit of X with the torus using the coordinate 
system. We denote this torus Tx C X . 

Fact 5.4. The torus Tx contains a point of an affine or projective 
space if and only if this is a point of X with all coordinates different 
from 0. 

The morphism j of character lattices is induced by the finite mor- 
phism from T(p(x(T,Z2)))3 = i^¥{x{T,Z2))f to Tp(x(r,Z2xZ2))- Due to the 
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discussion in the proof of Proposition 13.91 we also know that the mor- 
phism of ambient spaces does not depend on the tree, but only on the 
number of leaves /. Hence the vertical morphisms of lattices on the left 
hand side of Diagram 15.11 are the same for all trees with / leaves. 

5.2. Idea of the proof. The main reason for passing to tori is that 
we want to have a well defined dominant finite map. This allows us 
to take advantage of toric geometry. For example we know that the 
number of points in the fiber of the morphism of tori is equal to the 

' ~S 

index Ii of the image of j in (M^^q)^. 

For the projective ambient spaces the situation is a little bit different. 
The morphism / is not injective, so the corresponding morphism of tori 
is not surjective. We will show that the image of /in {M^lfis of finite 
index, say l2- It means that the corresponding morphism of tori is finite 
with each fiber having I2 elements. Moreover we will show that I2 = Ii- 
Hence we get the diagram: 



Ji(p(js:{T,Z2)))^ — ^ -itp(x(r,Z2xZ2)) 

where the horizontal maps are finite, etale of the same degree. 

This means that if we consider the morphism of projective ambient 
spaces, then the preimage of Tp(x(t,Z2xZ2)) is precisely T(p(x{t,Z2)))3- 
Hence any intersection results that hold for the binary model must also 
hold for the 3-Kimura model. In particular as Conjecture 13.101 holds 
for the binary model we obtain a set-theoretic version of Conjecture 
15.11 for the 3-Kimura model. 

By easy algebraic arguments we will also prove Conj ecture 15.11 scheme- 
theoretically for 3-Kimura model. 

5.3. Proof. Our first step will be to understand the morphism of pro- 
jective ambient spaces --^ r(W^^''^^). This is a well defined 
map on dense tori orbits. The map of tori corresponds to morphism 
of lattices / : M^q^^^ — )■ (M^g)'^. This morphisms depend only on the 
number of leaves, not on the tree. 

Each socket is an association of a group element to a leaf. Hence we 
can embed the group of sockets & in G'. We can also view the group 
© as a Z-module. This gives us group morphisms Ms — >■ (3 — >■ G'. 
The element of the basis of Ms indexed by a socket s is mapped to the 
socket s. 
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Example 5.5 (The case of the binary model and trivalent claw tree). 
Let us consider the tree K13 and the group Z2. We have got 4 sockets: 
(0, 0, 0), (1, 1, 0), (1, 0, 1), (0, 1, 1). By coordinate- wise action they form 
a subgroup of (^2)^. The lattice Ms is freely generated by four basis 
vectors 6(0,0,0)5 ^(1,1,0), e(i,o,i), e(o,i,i). The morphism Ms — )■ & maps 
e(a,b,c) to (a, b, c). Of course ke(^a,b,c) is mapped to k{a, b, c). For example 
3e(i,'i,o) is mapped to (1, 1, 0) +'(!, 1, 0) + (1, 1, 0) = (1,1,0). 

Lemma 5.6. We have an exact sequence of groups: 

Msf^^ {M^lf (Z2)'. 

The first morphism is given by f . The second is the group operation 
on sockets described abov^. 

Proof. It is clear that this is a complex. Let {b[)i>o be the basis of 
Mg^ corresponding to sockets. Let Si be the socket corresponding to 
b[. Moreover suppose that b'^ corresponds to the trivial socket, that 
is the neutral element of &. Let bi be the basis of M^q defined as 
bi = b'^ — 60 for ^ > 0. Note that an element (6^, b'p b'f?j is in the image 
of fl X X /g iff the corresponding three sockets Sj, Sj, Sk sum up to 
the neutral element of (3. Hence the elements of the form {bi,bi,0) = 
(b'^jb^jb^) — (6o,&o,fco) are in the image of /. We see that (26,, 0,0) = 
{bi,bi,0) + {bi,0,bi) — {0,bi,bi) is also in the image. Furthermore for 
any two sockets Si and Sj there exists a socket Sk ■= Si + sj such that 
{bi, bj, bk) is in the image of /. This reduces any element from (M^q)^ 
to an element (fcj, 0, 0) modulo the image of / or to 0. Hence any 
element is in the image if the XOR of all its coordinates is zero. □ 

Definition 5.7 (The kernel K). Let us consider the restriction K = 
Ki X K2X Ks c mI^o X mI^o X of the kernel of the morphism 

gx gx 9- 

Each character in ii' is a character of {T^^^Z2^)^, that is the trivial 

character when restricted to the product (Tp(x(Z2)))'^- Each such char- 
acter is a triple of characters of Tp^—za^ Each character of the triple is 
a quotient of monomials ^ of the same degree on the projective space 

F(W^'^). The polynomials mi — m2 spaiJlthe ideal of the toric variety 
P(X(Z2)). We want to view characters as functions. Hence we restrict 
our attention to (T^^^z^^Y. In the algebra of this torus the ideal of 

(Tp(x(Z2)))^ is generated by elements k — 1, where k ^ K. 



■In this case the second operation is often called XOR. 

'They do not only generate the ideal, but even span it as the vector space. 
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Definition 5.8 (The kernel D). For any tree T let be the kernel 
of the map h. 

The elements of D represent characters trivial on the projective 3- 
Kimura variety. In the setting of Subsection 13.21 we want to prove that 
sublattices for different trees T with / leaves generate the sublattice 
D^^''. The idea is to push the lattices D to (M^q)^ using the morphism 
/. Next we use the results on binary models to obtain the generation 
for f{D). Using properties of the image of / we are able to conclude the 
generation in M^q^^^. The following lemma enables us to restrict to the 
image of / instead of regarding whole lattice (M^q)'^. Algebraically it 
means that characters of Tp^^^zaxza^ trivial on the image of (T^^^z-^-^)^ 
are also trivial on Tp(x(T,Z2xZ2)) ^oi any tree T with / leaves. Hence 
geometrically the image of i^^^f)^^ contains Tp(x(r,Z2xZ2))- 

Lemma 5.9. For any tree T the kernel is a sublattice of the image 
off. 

Proof. It is enough to show that Ki x {0} x {0} C Im/. Suppose that 
m = 'Y^^ttibi G i^i, where each bi is as in the proof of Lemma 15.61 
Hence bi = {g\ — e, . . . , g} — e), where e is the neutral element of Z2 and 
g^ G Z2 are elements forming a socket. We know that g{m) = 0. In 
particular the coordinates of Me indexed by leaves are equal to zero. 
Let us fix k that is a number of a leaf 1 < k < I. Let us look at all 
coordinates indexed by pairs {k,q) where q G Z2. The restriction of 
Me to these coordinates is a free abelian group spanned by elements 
of Z2. Hence Ylii'^ii.dk ~ ^) = in the free abelian group generated 
formally by elements of Z2. Hence, a fortiori, Yli^iidk — e) = e where 
now the sum is taken in Z2. As the action in & is coordinate- wise we 
see that the image of m in (3, and hence in Z2, is the neutral element. 
Using Lemma [5.61 we see that m G Im/. □ 

Proposition 5.10. The index of the image of f in (M^q)"^ is equal to 

' 

the index of the image of g in (M^^q)'^. 

Proof. This is a consequence of the injectivity of g and Lemma \5M □ 

Corollary 5.11. Conjecture 15. 1\ holds set-theoretically. 

Proof. The index of the image of of / equals the degree of the finite 
map of tori. In particular we are in the situation of Diagram 15.21 
The corollary follows from the discussion at the beginning of Section 
[O □ 
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Now we will prove Conjecture 15.11 scheme-theoretically. Let Tq = 
Ki i. We consider such trees Tj that the ideal of Tp(x(T,Z2)) the sum 
of the ideals Tp(x(ri,Z2))- Let K^' be the kernel of g x g x g for the tree 
Ti. Let D^' be the kernel of h for the tree Tj. We know form Subsection 
1X2] that the lattices K'^' for z > span K'^°. 

Theorem 5.12. The lattices i^f^xZa y^r z > span Kq^""^''. Conjec- 
ture 15. il holds scheme theoretically. 

Proof. Let a e K^^''^^ We know that /(a) G so /(a) = ^ki, 

where ki G Kf^. Using Lemma [5.91 we can find /c- G xf'^^^^ such that 
/(/Cj') = /cj. This means that a — ^ /;;,• is in the kernel of /. In particular, 
as g is injective, a — ^ /c • belongs to every Kf^^^^, hence we obtain 
the desired decomposition. □ 

Remark 5.13. From Appendix \8.1\ it is enough to take two (particular) 
different i > to span Kq'^^'^^ , as it was in the case of binary model. 

6. Applications 

In this section we present a few applications of the main theorem. 
The basic result that we use is due to Marta Casanellas and Jesus 
Fernandez-Sanchez |CFS08j . It states that all biologically meaningful 
points are contained in the dense torus orbit of X(T, Z2 x Z2). In the 
main theorem we give a precise description of this orbit for any tree. 
This is sufficient for biologists. 

People dealing with applications are usually interested in trivalent 
trees. Let us motivate the use of other trees. The first, obvious reason 
is that they can appear (at least hypothetically) as right models of evo- 
lution. This however is a degenerate situation that is often neglected. 
The next subsection presents a different reason. 

6.1. Identifiability. Dealing with applications we are given a point P 
in the space of all possible probabilities Wl- The first question is for 
which trees this point can be realized. More precisely for which trees T 
we have an inclusion P G X(T, Z2 x Z2). We are interested in knowing 
if this is only one tree T or there are several possibilities. This is a first 
part of the identifiability problem. Hence Conj ecture 13 . 1 01 is a question 
about the locus of points for which the identifiability problem cannot 
be resolved at all. Of course a generic point that belongs to any of the 
varieties belongs to exactly one X(T, Z2 x Z2) with T trivalent. Much 
more is known about the identifiability of different models. For the 
precise results the reader is advised to look in |AR06j or [APRS] and 
the references therein. 
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In particular we see that points that belong to some X{T, Z2 x Z2) 
where T is not trivalent cannot identify the tree topology. Hence the 
question about the locus of these points, or equivalently about the poly- 
nomials defining such varieties may give some results for trivalent trees. 
However, as situation in Appendix 18.21 shows, the phylogenetic invari- 
ants of two varieties X(T, Z2) for two different trees, do not generate 
the ideal of the variety associated to their degeneration. 

The second, but equally important question about the identifiability 
is to give the description of the fiber ip~^{P). The biologist aim at 
distinguishing one point in the fiber. This would enable to identify not 
only the tree topology, but also corresponding probabilities of muta- 
tion. The algebraic setting allows us to give a description of this fiber. 
We assume that P is biologically meaningful, that is is contained in the 
dense torus orbit. Equivalently all coordinates of P after the Fourier 
transform are different from zero. We prefer to work up to multiplic- 
ity, that is regard the projectivization of denoted by ipp. The fiber 
'0p^(P) is contained in the dense torus orbit of n^(^e)- As this pa- 
rameter space is of the same dimension as image, we know that ijjp is a 
generically finite map. Moreover when restricted to dense torus orbits 
it is etale and finite. Hence each fiber is finite and contains the same 
number of points, independent from P. This number is the index of 
lattice Me in a saturated sublattice of Me- For 3-Kimura model this 
number can be calculated from |CFS08j . Of course we do not claim 
that all the points in the fiber have got a probabilistic meaning. We 
just prove that from the algebraic point of view there is always a fixed, 
finite number of possible candidates for transition matrices. 

6.2. Phylogenetic invariants. The main theorem gives an inductive 
way of obtaining phylogenetic invariants of any tree. It is an open 
problem if these invariants generate the whole ideal. It is proved how- 
ever that they give a description of all biologically meaningful points. 
The method is very simple. Suppose that we know the phylogenetic 
invariants for all trees with vertices of degree less or equal to d. Due to 
the results of [SS05j it is enough to describe the phylogenetic invariants 
for the claw tree Ki^^+i- To obtain them we just take the sum of two 
ideals. They are both associated to trees with the same topology. The 
tree has got two inner vertices vi and V2 of degrees 3 and d respectively. 
The difference between the ideals is a consequence of different labeling 
of leaves. For one tree the leaves adjacent to vi are labelled by 1 and 
2. For the second tree they are labeled 1 and 3. Notice that in fact we 
have to compute just one ideal. The second one can be obtained by 
permuting the variables. 
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7. Special cases 

7.1. Orbits. This section contains technical results concerning some 
orbits of the torus action that are not dense in the variety. Such orbits 
correspond to faces of the polytope associated to the variety. Each orbit 
is isomorphic to a torus. Its character lattice is the lattice spanned by 
points of the corresponding face of the polytope. In this notation the 
dense orbit corresponds to the polytope. 

Using the coordinates this description can be made more explicit. 
The points of the polytope correspond to coordinates of the affine (or 
projective) space. By choosing a face F we distinguish coordinates 
corresponding to points of this face. The orbit corresponding to F is the 
projection of the dense orbit onto the linear subspace. This subspace is 
given by a condition that all coordinates corresponding to points that 
are not in F are equal to zero. Hence closures of all orbits are given as 
intersections of the variety with linear subspaces corresponding to faces. 
The orbits themselves correspond to projections of the dense orbit onto 
these subspaces. In particular let Q be a point with all coordinates 
equal either to or 1. This point is contained in the toric variety if 
and only if points of the polytope corresponding to coordinates with 
the entry equal to 1 form a face. In case Q is contained in the toric 
variety it is the unique distinguished point of the orbit. It corresponds 
to the neutral element of the torus after identifying it with the orbit. 

We can decompose the affine space A*^ into tori Si = {0}'^ x 
by setting some coordinates equal to zero and requiring that others are 
different form zero. A toric variety intersected with such a subset is 
either empty or contains the unique distinguished point Qi = {0}'' x 
|]^jr!.-fc_ j£ jg nonempty it is a torus. If we intersect a few toric 
varieties with Si the situation is a little bit more complicated. We 
can get an empty set if and only if one of the varieties has an empty 
intersection with Si. Otherwise the intersection contains the point Qi. 
It might however be reducible. 

To be even more precise let M be the character lattice of the torus 
acting on A". Let Mj be the character lattice of affine toric varieties 
Xi C A"-. Let ipi : M ^ Mi be the surjective morphism corresponding 
to the inclusion and let Ki be its kernel. There is a natural projection of 
A" D (C*)*^ — )■ Si- This induces an inclusion of lattices M' C M, where 
M' is the character lattice of the torus acting on Si. We may restrict 
(fi and Ki to M' obtaining 93 • and K'^. The question whether f]XinSi 
is irreducible is equivalent to the question if IJ K- is saturated in M. 
If not, its index is equal to the number of connected component^ 



Each is irreducible and isomorphic to a torus. 
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Moreover the saturation of Kq := U-ft'j' gives characters trivial on the 
distinguished component of the intersection that contains Qi. Hence 
M/Kq is the character lattice of this component. 

To check Conj ect ure 13 . 1 1 set theoretically we may check it separately 
on each S^. This approach is quite standard, see e.g. |ES96] . For a given 
Si we have a few different possibilities. 

Case 1) The intersection is empty. 

The first possibility is that the intersection f]rp_^j^_^ ^ X{T, G) D Si is 
empty. It follows that X{T,G) fl Si is empty for some T ^ -ft'i,/. As 
X{Ki^n,G) C X{T,G) the intersection X{Ki^n,G) fl Si is also empty 
hence Conjecture 13.101 holds on Si. 

Case 2) The intersection is reducible. 

In order to prove the Conjecture 13. 101 one has to exclude this possi- 
bility. 

Case 3) The intersection is nonempty and irreducible. 

Here we have to exclude the case of the strict inclusion X{Ki^n, G) fl 
Si C f]rp-^x^ ^ ^{T, G) nSi. As both intersections are isomorphic to tori 
it is enough to compare their dimensions. 

7.1.1. Special faces - type 1. Unfortunately we are able do conclude 
only for some special tori Si. Moreover the reasoning presented in this 
section is often very technical. We would much appreciate a uniform 
approach that would enable the proof of Conjecture 13.101 

First let us consider faces -F(e,g) of the type h*eg-^{x) = 1, where 
h{e,g) G Me is a basis vector. This face contains networks that associate 
g to edge e. Let us consider the claw tree Ki^ and any relation R 

between the vertices of the face F^^^y Let T be a tree with two inner 
vertices. One of the vertices Vi is of degree 3 and the second one is of 
degree I — 1. The edge corresponding to e is a leaf adjacent to Vi. 

(7.1) 




One can easily see that the relation R lifts up to the relation between 
the vertices of the polytope associated to T. Hence X{Ki^i,G) and 
X{T, G) are scheme theoretically equal on the closure of Si. Faces 
contained in some F(^e,g) will be called faces of type 1. 



TORIC GEOMETRY OF THE 3-KIMURA MODEL FOR ANY TREE 21 



7.1.2. Special faces - type 2. Now let us assume that G = Z2 x Z2. We 
will give a description of faces that contain a diagonal. We say that a 
face contains a diagonal if for some two vertices, it is a minimal face 
containing them. Hence all edges are included. Examples of polytopes 
that do not contain a diagonal are simplices or piramids. 

Let us consider two networks ni and n2 on a claw tree. We will 
use the same notation for networks and corresponding vertices of the 
polytope. 

Proposition 7.1. The minimal face containing rii and 77-2 contains 
precisely those networks n, such that for each edge e we have either 
n(e) = ni(e) or n(e) = n2(e). 

Proof. It is obvious that this set of networks forms a face. If we consider 
any network n with the property above, we may define a new network 
n' by the property: 

n'(e) = 1'^^*^^'' if n(e) = n2(e) 
|n2(e) if n(e) = ni(e) 

For corresponding vertices we have n + n' = ni + n2- Hence n and n' 
must belong to to the minimal face containing ni and 722. □ 

For Si corresponding to such faces one can also prove Conjecture 
13.101 The proof is technical and amounts to decomposing any network 
in the face plus kni for G N into sum of networks that do not differ 
much from ni. 

7.2. Irreducibility. We finish with a remark about the reducibility of 
the intersection of varieties associated to different trees. We prove that 
for the 3-Kimura model if the intersection is reducible, then it must 
have at least three components. 

The space We is the regular representation of 9T. The variety X{T, G) 
is invariant with respect to its action. We also know that each compo- 
nent of the intersection f]X{T, Z2 x Z2) different from X{Kii, Z2 x Z2) 
is in a hyperplane section. Suppose that Conjecture 13. 101 does not hold 
set theoretically. The action of 91 permutes the components. It also 
acts transitively on variables. In particular it sends a hyperplane sec- 
tion to a different hyperplane section. Hence there must be at least 
several components in the intersection. 

8. Appendix 

8.1. Let us fix a number of leaves /. We claim that for special two 
trees Ti and T2 the scheme-theoretic intersection X(Ti, Z2) nX(T2, Z2) 
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equals X{Ki^i,Z2). We number the leaves from 1 to /. The trees Ti 
and T2 are isomorphic as graphs but have different leaf labeling. The 
topology of the trees is the same as the one described in Picture 17.11 
For the tree Ti the leaves adjacent to vi have got numbers 1 and 2. 
For the tree T2 they are numbered 1 and 3. The ideal of the variety 
associated to a tree for the group Z2 is always generated in degree 
2 |CP07j . |SS05] . Hence the generators of the ideals are of the form 
nin2 = n^n^ where rij for 1 < i < 4 are coordinates corresponding 
to networks. A network is an association of group elements to edges. 
In can be represented by a vector, whose entries are group elements, 
indexed by edges. A product of coordinates corresponding to networks 
can be represented by a matrix with columns representing the networks. 
Of course we can permute the columns in such a matrix. An equality 
of two products can be represented by two matrices. In fact each 
binomial equality corresponds to a pair of matrices {Mq,Mi) whose 
columns represent networks and rows are the same up to permutation. 
Hence each generator of the ideal of X(i^i ;,Z2) is represented by a 
pair of 2 X Z matrices with entries from Z2. Moreover the sum in each 
column is the neutral element and rows of both matrices are the same 
up to permutation. As we can permute columns of each matrix we may 
assume that the first rows of both matrices coincide. Let us consider 
any such generator (Mo,Mi) in the ideal of X{Ki^i,Z2). 

First suppose that the entries in the first row are the same, that is 
either 00 or 11. Then the relation holds both for X{Ti) and X{T2). 
Hence we may suppose that the first row is 01 or 10. The same reason- 
ing holds for the second and third row. Hence all three rows in both 
matrices are either 01 or 10. If the second (resp. third) rows are the 
same in both matrices then the relation holds for X{Ti) (resp. X{T2)). 
So the only possibility left is that the second and third rows of Mi 
are the negation of the second and third rows of Mq. In this case the 
relation does not hold in any X{Ti) but we can generate it. We con- 
sider a matrix M that is equal to Mq with the first two rows permuted. 
The pair (Mo,M) represents a relation in X[Ti). Moreover the pair 
(M, Ml) represents a relation in X{T2). 



8.2. We will prove that a scheme theoretic intersection X(Ti,Z2) fl 
X(T3,'Ia2) does not have to be equal to X{Ki^i,Z2) even if Ki i is the 
only tree smaller then Ti and T3. We consider the case of five leaves 
/ = 5. The tree Ti is the same as in the previous subsection. The tree 
T3 is isomorphic, with two distinguished leaves labeled with 4 and 5. 
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We consider the relation given by a pair of matrices: 



■ 1 


" 




■ 1 


" 





1 




1 






















1 







1 


1 










1 



This corresponds to a generator of the ideal of X(i^i^5,Z2). Consider 
any relation involving the first matrix and some other matrix M for 
X{Ti) or X{T-i). One can see that the first two rows of M must be 
negations of each other and the third one is 00. Hence it is impossible 
to generate the relation above. 
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